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ABSTRACT 

In its fourth year. Program Assistance for Neophytes 
(Project PAN) served 455 students at 2 high schools. The program 
provided support services and supplemental instruction to students of 
limited English proficiency (LEP) as well as English-proficient (EP) 
students. The project provided LEP students with a transitional 
period of bilingual education. Native language arts (NLA) classes 
were available for EP students deficient in their native languages. 
The project targeted students whose native languages were Chinese, 
Korean, Vietnamese, or Urdu, and provided 

English-as-a-Seconrt-Language (ESL) and NLA instruction (in Chinese 
and Korean), as well as bilingual mathematics, science, social 
studies, and career classes. Staff and curriculum development 
activities and ESL for parents were also offered. Three mini-ore jects 
functioned within the larger project, two pairing LEP and EP students 
for peer tutoring and extracurricular exntivities, and one providing 
computer instruction. Objectives in ESL instruction, parent 
involvement, and attendance and one objective in NLA were met. 
Program strengths include the instructional components, curriculum 
de^'elopment , and peer pairing. The major weakness was a lack of 
supplementary services in the native language for students who were 
not Chinese or Korean. (Author/MSE) 
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EXECUTIVE SUMMARY* 

• Project PAN was fully implemented. During the 1988-89 
school year, participating students received 
instruction in English as a Second Language (E.S.L.)# 
Native Language Arts (N.L.A.)/ bilingual content area 
subjects, and career education. The program offered 
support services to students, and staff development, 
curriculum development, and parental involvement 
activities. 

Project PAN met its objectives in E.S.L. , N.L.A., 
attendance, and parental involvement. It met one of 
its objectives in content area courses but did not 
supply data to assess the other. 

Program Assistance for Neophytes (Project PAN) completed its 
fourth year, the first of a two-year extension of an Elementary 
and Secondary Education Act (E.S.E.A.) Title VII funding cycle. 
The project served approximately 460 students at John Bowne and 
Newton High Schools in Queens. It provided support services and 
supplemental instruction to students of limited English 
proficiency (LEP students) and to those who were English 
proficient (EP students) . Project PAN provided LEP students with 
a transitional period of bilingual education to them keep abreast 
of mainstream students in the content areas, classes in Native 
Language Arts (N.L.A.) were available for EP students who were 
deficient in their native language. 

The project targeted students whose native language was 
Chinese, Korean, Vietnamese, or Urdu. The project provided 
instruction in English as a Second Language (E.S.L.); N.L.A. in 
Chinese and Korean; bilingual mathematics, science, and social 



*This report is based on the "Program Assistance for Neophytes 
(Project PAN), 1988-89" prepared by the OREA 
Multicultural/Bilingual Education Evaluation Unit. 
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studies; and career classes. The project also offered staff and 
curriculum development activities as well as E.S.L. classes for 
parental involvement. Three mini-projects functioned within 
Project PAN: Projects Mimic and Chum paired LEP with EP students 
to facilitate English language learning through peer tutoring and 
extracurricular activities, and Project Compute provided students 
with computer instruction. 

The project proposed objectives in E.S.L. , N.L.A., parental 
involvement, and student attendance. One E.S.L. objective, 
related to improvement on the Language Assessment Battery, was 
modified by OREA because it could not be assessed as proposed. 
This modified objective was met. The other objective was related 
to performanca on teacher-made tests, and it was also met. One 
N.L.A. objective was achieved, but OPJIA could not assess the 
second because of lack of data. The project met its objectives 
in parental involvement and attendance. 

The strongest aspects of the program were its instructional 
components, curriculum development, and peer pairing of LEP and 
EP students. The weakest area was the lack of supplementary 
servicefi in the native language for Asian students who were not 
Chinese or Korean. (This was also noted in last year's report.) 

The conclusions, based on the findings of this evaluation, 
lead to the following recommendation: 

• If funds permit, offer bilingual instruction for the 
Vietnamese and Urdu speakers in the program. 
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second because of lack of data. The project met its objectives 
in parental involvement and attendance. 

The strongest aspects of the program were its instructional 
components, curriculum development^ and peer pairing of psP and 
EP students. The weakest area was the lack of supplementary 
services in the native language for Asian students who were not 
Chinese or Korean. (This was also noted in last year's report.) 

The conclusions, based on the findings of this evaluation, 
lead to the following recommendations 

If funds permit, offer bilingual inistruction for the 
Vietnamese and Urdu speakers in the program. 
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I. INTRODUCTION 



This report documents the Office of Research, Evaluation/ 
and Assessment's (OREA's) evaluation of the Title VII funded 
project, Program Assistance for Neophytes (Project PAN) . Project 
PAN, In Its fourth year of funding, was a bilingual program that 
provided supplemental Instruction and support services for 
students of limited English proficiency (LEF students) and of 
English proficiency (EP students) . Its primary goal was to 
provide a transitional period of bilingual education to Insure 
that the progress of LEP students paralleled that of mainstream 
students. Classes In Native Language Arts (N.L.A.) were 
available for those who were deficient In their native language. 
The project provided English as a Second Language (E.S.L.) 
classes for Interested parents as well as curriculum and staff 
development activities. 

HISTORY OF THE PROGPAM 

The final evaluation reports of 1986-87 and 1987-88 give a 
complete description of the history of the program. For program 
Implementation and outcomes of objectives for previous years, see 
the evaluation reports for those years. 

SETTING 

The project operated at John Bowne and Newtown High Schools 
in Queens. The areas in which the schools were situated had a 
large number of Chinese, Vietnamese, Korean, Indian and 
Pakistani residents. 
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PARTICIPATING STUDENTS 

Program students came primarily from China, Hong Kong, 
Taiwan, Vietnam, Korea, India, and Pakistan. Among the Chinese, 
Vietnamese, and Pakistani groups there were many recent 
immigrants. Overall, more than 90 percent of the students had 
less than two years of education in the United States. Over half 
of them were over-age for their grade. (See Table 1.) Many 
students from the rural areas of mainland China had received 
limited education in their own language. Project students from 
Taiwan had tae strongest educational backgrounds. According to 
the project director, most of the students lived in extremely 
crowded conditions. In most families, both parents worked, and 
55 percent of the students worked after school. 

The program staff included a project director, a resource 
specialist/ guidance counselor, a secretary, two educational 
assistants, a family assistant, and an educational associate. 
All worked full-time. The project director and the resource 
specialist both had Ph.D.s and were fluent in several dialects of 
Chinese. The secretary and the two educational assistants had 
master's degrees and weru fluent in Chinese. The family 
assistant and the educational associate both had bachelor's 
degrees; one was fluent in Vietnamese, the other in Chinesie. 

The project director, who was also the director of Project 
BEACON (another Title Vll-funded program) , was responsible for 

2 

er|c 12 



TABLE 1 

Number of Program Students by Age and Grade* 



Age Grade 9 Grade 10 Grade 11 Grade 12 Total 




2£ _Q 2 S Ifl ZL. 

21 Q Q 2 2 i. 

22 Q 2 Q 1 3_ 

lOm I 2i I lafi I 122 I 62 I 455" 



Over-Age Students 



Number 


22 


110 


80 


36 


248 




29t? 


61.1 


5§t1 


58.1 


§4t5 



Note: Framed boxes indicate expected age range for grade. 
*As of June 1989. 

"Data were missing for eighteen students. 

• Fifty-five percent of program students were 
over-age for their grade. 
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overall supervision. The resource specialist had a degree in 
counseling psychology and p.'ovided curriculum development, 
counseling, and guidance to Chinese-speaking students and also 
acted ds an interpreter for other administrative personnel. The 
resource teachers provided computer-assisted instruction and 
assisted in carricul jn development. The family assistant was 
responsible for family contact and translating. The educational 
associate worked on curriculum development on a per-session 
basis. The secretary provided clerical services for the project 
at the central office, located at John Bowne High School. 

DELIVERY OF SERVICES 

Students received instruction in E.S.L. ; N.L.A. where 
available; bilingual and E.S.L. mathematics, science, and social 
studies; apd E.S.L. career classes. In the mini-projects Mimic 
and Chum, Project PAN paired LEP with EP students to facilitate 
English-language learning through peer tutoring and 
extracurricular activities; the mini-project Compute provided 
computer instruction. 

Additional services included counseling in Chinese, 
Vietnamese, and Korean; a monthly newsletter and E.S.L. classes 
for parents; and three culture clubs (Korean, Chinese, and 
Asian) , which organized sporting events and other activities and 
published a club magazine. The project also engaged in a wide 
range of staff and curriculum development activities. 
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REPORT FORMAT 

This report is organized as follows: Chapter II describes 
the evaluation methodology; Chapter III describes the 
implementation of the project and assesses its accomplishment of 
implementation objectives; Chapter IV presents an analysis of 
the student outcome data; Chapter V offers conclusions and 
recommendations based upon the results of the evaluation. 
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II. EVALUATION METHODOLOGY 



EVALUATION QUESTIONS 

The evaluation assessed two major areas: program 
Implementation and outcomes. Evaluation questions Included the 
following: 

Process/Implementation 

• Did the project select students for program 
participation according to specific criteria? 

• Did the project Implement Instructional activities for 
developing English Language proficiency as proposed? 

• Did the project Implement Instructional activities for 
developing native language skills as proposed? 

• Did the project Implement bilingual and E.S.L. 
Instruction In the content areas of mathematics, 
science, and social studies? 



Outcome 



What was the average Normal Curve Equivalent (N.C.E.) 
gain on the Language Assessment Battery (LAB)? 

What percent of program students passed their courses 
In E.S.L.? 

What percent of program students passed their courses 
In N.L.A.? 

What percent of program students passed their courses 
In mathematics, science, social studies, and career 
classes? 

How did the passing rate on translated Regents 
examinations In science and social studies of prograic 
students compare with that of mainstream students? 

How did the attendance rate of program students compare 
with that of mainstream students? 



ERIC 



16 



EVALUATION PROCEDURES 

Sample 

An OREA field consultant visited both program sites. Ke 
observed four classes, and interviewed the project director, the 
resource specialist/ guidance counselor, and the principals at 
both sites. OREA provided a student data form for each student. 
The project returned 384 completed forms in the fall and 445 in 
the spring. 

Instruments 

OREA developed an interview and observation schedule for the 
use of the field consultant and a questionnaire for the project 
director. Project personnel used OREA-developed data retrieval 
forms to report student demographic, attendance, and achievement 
data . 

Data Collection 

The consultant interviewed school and program staff and 
observed classes during a four-month period from February to May 
1989. OREA distributed the project director questionnaire and 
student data forms to the program director in January and April 
and collected them at the end of February and June. 

Pat?^ Analygjg 

OREA used the Language Assessment Battery (LAB) to assess 
improvement in English proficiency. Project PAN students were 
tested at grade level each spring. Students' raw scores were 
converted to Normal Curve Equivalent (N.C.E.) scores, which have 



multiple advantages over otner scoring methods. They are 
standard, normalized, and form an equal-interval scale. 
("Standard" indicates that the unit of measurement is a fraction 
of the standard deviation of the original diswribution of raw 
scores; "normalized" refers to the fact that the scale is 
adjusted for the norm group so that its distribution has the 
shape of a normal distribution; and "equal interval scales" allow 
for legitimate aggregation or averaging of scores.) Project 
students* N.C.E.s indicated their standing in relation to the 
national average of 50. 

To assess the significance of students* achievement in 
English, OREA computed a correlated j;-test on LAB N.C.E. scores. 
The ^-t est determined whether the difference between the pre- and 
posttest scores was significantly greater than would be expected 
by chance variation alone. 

To insure representative achievement data, OREA included 
only those students who had been in the program for at ?east five 
months and had attended classes for at least 100 school days. 
OREA extrapolated to estimate full-year scores of late-arriving 
and early-exiting students. 

Limitations 

Since all LEP students are entitled to receive bilingual and 
E.S.L. services, OREA was unable to select an equivalent control 
group. However, the use of two sets of da^.a, as outlined above, 
sert/ed in lieu of a control group. 
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III. EVALUATION FINDINGS: IMPLEMENTATION 



Project PAN provided students with Instruction in E.S.L. , 
N.L.A., content area subjects taught bilingually or with an 
E.S.L. approach, and career classes. The project provided staff 
development, curriculum development, and E.S.L. classes for 
parents. Mini^projects provided peer tutoring and computer 
instruction. 

STUDENT PLACEMENT AND PROGRAMMING 

The project targeted Chinese, Vietnamese, Korean, and Urdu 
speakers classified as LEP and a small number of EP students who 
wished to learn the language and culture of their ethnic group. 
The project staff chose the LEP participants on the basis of LAB 
test scores and school records. Project staff and school 
principals recommended EP students for participation. 

Students were programmed for a two-year sequence of bilingual 
content area and E.S.L. courses, usually beginning in the nii^th 
grade. The project director noted that twelve percent of the 
project students were mainst; eamed in the fall of 1988 and had 
done (luite well. After two /ears in the program, if the student 
still scored below the twenty-first percentile on the LAB, he or 
she was placed in the foreign students program, which offered 
special classes taught in simplified English in each subject 
area. 

Students deficient in their native language took a two-ye^r 
sequence of N.L.A. courses. 

9 
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INSTRUCTIONAL ACTIVITIES 

Instructional activities included E.S.L., N.L.A., content 
area subjects, and career education classes. 

English as a Second Language 

The project provided four levels of E.S.L. classes 
(elementary, intermediate, advanced, and transitional) . The 
advanced and transitional levels were English for Foreign 
Students classes; they were available to both EP and LEP 
students . 

At John Bowne High School, the field consultant observed an 
E.S.L. class. Students wrote homework dialogue questions on the 
board. The teacher then referred the class to their books and 
asked them to practice the dialogutds with their partners for five 
to ten minutes. As they spoke with each other, the teacher moved 
among them, listening, offering feedback, and participating. He 
then listed appropriate phrases covering a variety of situations. 
The students worked at their desks on material similar to the 
boardwork. After the teacher reviewed the work, the students 
practiced the dialogues in pairs. 

Native L anguage Arts 

The project provided elementary, intermediate, and advanced 
levels of N.L.A. classes in Chinese and Korean for both native 
and non-native speakers at the two high schools. 
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Content Area Subjects 

The project provided bilingual content area classes. It 
offered global history, American history, government, general 
science, and biology in Chinese, and global history and biology 
in Korean. The field consultant observed three of these classes. 

Project PAN also offered mathematics, science, social 
studies, speech, and typing courses, all taught with an E.S.L. 
methodology. 

The Unified Science class at J-^hn Bowne High School began 
with the paraprofessional collecting the previous day's homework. 
The teacher wrote the new homework assignment on the board, asked 
if there were any questions, and began the day's topic, the 
geographical North Pole. After giving some background, she wrote 
'*How do we describe the magnetic nature of the Earth?" on the 
board. She used English to write names and dates. She paused in 
her lecture to answer questions, restated and summarized the main 
points in English, and asked students to respond in English. 
During this time, the paraprofessional looked over the homework 
papers . 

In the biology class at Newtown High School, the teacher 
began by passing out a list of biological terms with their 
Chinese translations* He had a few students pronounce the terms 
aloud. He discussed, in English, the structure of seeds and 
fruits. He took out some actual seeds and fruits and 
demonstrated the differences between them. A few students went 
to the board to diagram the structure of a lima bean. The 

♦ 
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teacher summarized, listing parts, and asked a student to define 
the function of each part on the board (in Chinese) . The class 
went on to another subject and ended with a di&^cussion of the 
homework assignment. 

The OREA consultant observed a class in government at the 
same school. The main thrust of the session was how candidates 
win elections. The format of the lesson was question-and-*answer, 
mostly in Mandarin and Cantonese. Toward the end of the session, 
students wrote key concepts and vocabulary on the board in 
English and Chinese. 

NQNINSTRUCTIONAL ACTIVITIES 

Project PAN provided support services for students and 
staff, curriculum development, and parental involvement 
activities. 

Support Services 

Counseling . The project provided counseling in Chinese, 
Vietnamese, and Korean. This included orientation for new 
students, academic evaluations, and college and career 
advisement. It provided translation services for school 
personnel who worked with these students. 

Peer Tutoring . Project PAN established the mini-projects 
Mimic and Chum to provide peer tutoring. EP students had the 
chance to practice their chosen language while helping LEP 
students with English and content area subjects. Six staff 
members, three at each site, supervised each tutoring session. 

12 



students responded with enthusiasm, and one-third of them 
participated. 

Field Trips . Participating students took field trips to the 
Brooklyn Botanical Gardens, the New York Aquarium, the old 
Bethpage Village Restoration, and the Planting Fields Arboretum 
at Oyster Bay. Students also attended a career conference at 
Queensborough Community College. 

Culture Clubs . Students could participate in the activities 
of culture clubs (Chinese Culture Club, Korean Culture Club, 
Asian Culture Club) . These organized various sporting and 
cultural events. 

Computer Skills . A third mini-project. Project Compute, 
made a computer lab or the equivalent available to Project PAN 
students. Assisted by both a bilingual paraprofessional and a 
content area teacher, students could work on mathematics, 
science, and social studios. 

Staff PevelQpmen^ 

Staff development activities included attendance at 
workshops on child abuse and neglect, cultural conflicts, and 
acculturation. Staff also attended a New York Multifunctional 
Resource workshop. 

Curriculum Development 

Project staff developed computer software for the biology 
and global history Regents examinations. They completed a 

13 
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Newtown High School Handbook In Chinese, and were in the process 
of completing such a handbook in Korean and Vietnamese. 

Parental Involvement 

The program objective for parental involvement was: 

*. The program will offer E.S.L. classes for parents once 
a week. 

The project offered four weekly hour-long E.S.L. classes for 
families of project students. Over 200 adults attended. Project 
PAN met its parental involvement objective. 

The project also sent families monthly newsletters in 
Chinese, Korean, and Spanish. Parents attended an introduction 
to and discussion of available financial aid for college, an 
International Food Festival, a career workshop. Parents 
Day/Night, and meetings of the Parents' Advisory Council (PAC) . 
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IV. EVALUATION FINDINGS: OUTCOMES 



INSTRUCTIONAL ACTIVITIES 

Project PAN proposed instructional objectives in E.S.L. , 
N.L.A. , content area subjects, and career classes. 

English as a Second Language 

The evaluation objectives for English language development 

were: 

• As a result of participating in the program, E.S.L. 
students will master an average of one E.S.L. 
curriculum objective per 20 days as indicated on the 
appropriate level of the LAB. 

• At least 70 percent of the students will score at or 
above the passing criterion of 65 in second language 
classes. 

Since the LAB is not a criterion referenced test, it was 
impossible to evaluate the first E.S.L. objective as stated. In 
lieu of this, OREA looked at pretest/posttest differences on the 
LAB to determine whether students had acquired a significant 
amount of skill in English over the course of the program year. 
Complete LAB scores were available for 226 students. (See Table 
2). The mean N.C.E. gain was 6.4 (s.d.: 10.8), which was 
statistically significant (c<.05). 

Students* performance in teacher-made tests was also very 
good: 93 percent of 262 students in the fall and 88 percent of 
260 students in the spring achieved the passing criterion of 65 
percent. The project met its second E.S.L. objective. 
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TABLE 2 



Pretest/ Postt est N.C.E. Differences on the 
Language Assessment Battery by Grade 



Number Ptet^gtZ Posttest Difference ^ 

Grade of Students Mean S.D. Mean S.D. Mean S.D. value 



9 


36 


9.6 


12.1 


22.6 


x^.2 


13.0 


10.1 


7.71* 


10 


99 


13.7 


11.3 


18.8 


11.4 


5.1 


. .3 


7.03* 


11 


64 


17 . 3 


11.4 


23 .8 


13.9 


6.5 


14.6 


3.58* 


12 


27 


15.9 


7.9 


18.0 


7.8 


2.1 


8.0 


1.38 


Hiah School 


















John Bowne 


101 


17.2 


11.5 


21.2 


12.5 


4.0 


7.4 


5.49*. 


Newtown 


125 


12.0 


10.6 


20.3 


11.8 


8.3 


12.5 


7.42* 


TOTAL 


226 


14.3 


11.3 


20.7 


12.1 


6.4 


10.8 


8.97* 



* E<.05 

• Students in all grades but twelfth, and overall, made 
statistically significant gains. 

• Students at both schools made statistically significant gains. 
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Native Language Art s 

The evaluation objective for the development of native 
language skills was: 

• At least 70 percent of the students will score at or 
above the passing criterion of 65 in native language 
classes. 

Ninety-eight percent of 203 students who took N.L.A. classes 
in the fall and 94 percent of 214 students who took them in the 
spring, achieved the passing rate of 65 percent. Therefore, the 
project met its N.L.A. objective. 

Content Area Subjects and Career Classes 

The evaluation objectives for content area subjects and 

career classes were: 

« At least 70 percent of the students will score at or 
above the passing criterion of 65 in mathematics, 
science, social studies, and career classes, as 
indicated by final course grades. 

• Program students will have equal or higher passing 
rates than mainstream students in translated Regents 
examinations in science and social studies. 

Project students surpassed the passing criterion in every 
area. The percentage of students passing ranged from 83.1 in 
mathemai:ics in the fall to 100 percent in career courses in the 
spring, (see Table 3.) T>roject PAN met its first content area 
and career classes objective. 

OREA could not assess the second objective because the 
project did not submit data. 
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TABLE 3 



Passing Rates in Content Area Subjects 




Fall 




Sorina 






Number of 


Percent 


Number of 


Percent 


Subject 


Students 


Passing 


Students 


Passing 


Mathematics 


71 


83.1 


31 


93.5 


Science 


220 


93.2 


164 


87.8 ' 


Social Studies 


232 


94.8 


191 


86.4 


Career Courses 


43 


95.3 


17 


100.0 



More than 80 percent of the students achieved the passing 
rate in all areas. 
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NONINSTRUCTIQNAL ACTIVITIES 

The project posed a nuninstructional outcome objective in 
attendance. 

Attendance 

The evaluation objective for attendance was: 

• As a result of participating in the program, students' 
attendance will be significantly higher than the 
attendance of mainstream students. 

OREA used a latest for the significance of a proportion to 

assess this objective. This procedure determines whether the 

difference between one proportion (the program's attendance) and 

a standard proportion (the school's attendance) is greater than 

can be expected by chance variation. The attendance rate for 

program students at John Bowne High School was 97.1 percent and 

at Newtown High School was 96.6 percent. Comparisons with the 

schools' attendance of 86.7 and 85.7 yielded ^=3.85 and £-3.86 

respectively, which were statistically significant (£<.05). The 

project met its attendance objective. 
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V. CONCLUSIONS AND RECOMMENDATION 



Project; PAN identified, selected, and progranuned IiEP 
students into a series of E.S.L., N.L.A., content area, and 
career education classes. The project also offered services to a 
small number of EP students interested in learning the language 
of their native culture. Project PAH provided support services 
to students; staff and curriculum development activities; and 
parental Involvement activities, including E.S.L. classes, 
workshops, and PAC and other meetings. The project also 
disseminated a monthly newsletter in Chinese, Korean, and 
Spanish. 

Project PAN met its objectives in E.S.L. by significantly 
improving students' skills in English, as indicated by LAB and 
teacher-made test scores. The project also met its N.L.A. 
objectives, since at least 94 percent of the program students 
achieved the passing criterion of 65 percent. It met one 
objective in the content areas (for 70 percent of the students to 
achieve passing grades) but did not submit data for OREA to 
assess the second objective. 

Project PAN met the objective in parent involvement calling 
for the provision of E.S.L. classes for families of participating 
students. It also met the attendance objective, since program 
students had a higher attendance rate than students schoolwide. 

An interesting feature of the project was its use of mini- 
projects—Projects Mimic, Chum, and Compute. These allowed LEP 
and EP students to help each other by addressing each other's 
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strengths and weaknesses through peer tutoring, and the computer 
assisted students with language skill development and content 
area courses* 

Any shortcoming that can be found with the project was 
related to its predominantly Chinese and Korean focus* While 
there were far fewer speakers of languages other than Chinese and 
Korean in the program, they would have benefited from bilingual 
counseling and content area instruction. 

The conclusions, based on the findings of this evaluation, 

lead to the following recommendation: 

• If funds permit, offer bilingual instruction for the 
Vietnamese and Urdu speakers in the program. 
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